Minimal Solutions for Panoramic Stitching with Radial Distortion

This paper presents a solution to panoramic image stitching of two images with coinciding optical centers, but unknown focal length and radial distortion. The algorithm operates with a minimal set of corresponding points (three) which means that it is well suited for use in any RANSAC style algorithm for simultaneous estimation of geometry and outlier rejection. Compared to a previous method for this problem, we are able to guarantee that the right solution is found in all cases. The solution is obtained by solving a small system of polynomial equations. The problem of dealing with cameras with various forms of nonlinear distortions in computer vision is not new, but traditionally these lens effects have only been incorporated at the bundle adjustment stage. However, in situations with more than very mild distortions, it might be necessary to account for non-linear effects already at the RANSAC stage. Recently a number of contributions have been made on the efficient estimation of geometric relations in computer vision along with radial distortion [1, 4]. A direct inspiration for this work is the two-point algorithm for estimating rotation and focal length by Brown et al. [2]. This algorithm does however not handle any distortion and we show that for non-standard lenses, this might be insufficient. Most closely related to our approach is the work by Jin [6]. Jin formulates the same problem as we do. However, Jin uses an iterative optimization based scheme which is not guaranteed to find the right solution. We consider a setup with two cameras P1 and P2 with a common focal point. We fix a coordinate system where the common focal point coincides with the origin and such that the first 3x3 part of the matrix P1 is the identity. Moreover, we have a set of world points {X j} and corresponding image projections {u1 j} and {u2 j}. We assume square pixels, zero skew and centered principal point [5] and thus obtain the following relations